Novel Multi-layer Non-negative Tensor Factorization with Sparsity Constraints

In this paper we present a new method of 3D non-negative tensor factorization (NTF) that is robust in the presence of noise and has many potential applications, including multi-way blind source separation (BSS), multi-sensory or multi-dimensional data analysis, and sparse image coding. We consider alphaand beta-divergences as error (cost) functions and derive three different algorithms: (1) multiplicative updating; (2) fixed point alternating least squares (FPALS); (3) alternating interior-point gradient (AIPG) algorithm. We also incorporate these algorithms into multilayer networks. Experimental results confirm the very useful behavior of our multilayer 3D NTF algorithms with multi-start initializations. 1 Models and Problem Formulation Tensors (also known as n-way arrays or multidimensional arrays) are used in a variety of applications ranging from neuroscience and psychometrics to chemometrics [1–4]. Nonnegative matrix factorization (NMF), Non-negative tensor factorization (NTF), parallel factor analysis PARAFAC and TUCKER models with non-negativity constraints have been recently proposed as promising sparse and quite efficient representations of signals, images, or general data [1–14]. From a viewpoint of data analysis, NTF is very attractive because it takes into account spacial and temporal correlations between variables more accurately than 2D matrix factorizations, such as NMF, and it provides usually sparse common factors or hidden (latent) components with physical or physiological meaning and interpretation [4]. One of our motivations is to develop flexible NTF algorithms which can be applied in neuroscience (analysis of EEG, fMRI) [8, 15, 16]. The basic 3D NTF model considered in this paper is illustrated in Fig. 1 (see also [9]). A given tensor X ∈ RI×T×K + is decomposed as a set of matrices A ∈ ? On leave from Warsaw University of Technology, Dept. of EE, Warsaw, POLAND ?? On leave from Institute of Telecommunications, Teleinformatics and Acoustics, Wroclaw University of Technology, POLAND 2 A. Cichocki et al.